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ABSTRACT 

The  recent  progress  in  artificial  intelligence  based  machine  learning  techniques  has  renewed  interest  in 
building  and  designing  the  interactive  characters.  To  achieve  optimal  performance  many  techniques  have  been  used 
along  with  the  neural  networks,  which  are  prepared  end  to  end  in  tasks  such  as  object  recognition  video  games  and 
board  games.  In  this  paper,  a  review  of  current  literature  is  carried  out  where  the  challenges  in  real  time  strategy  game 
and  the  tasks  in  a  real-time  environment  are  explored.  The  comparative  analysis  is  made  on  different  AI  based  machine 
learning  techniques  to  comprehend  the  performance  issues  and  other  challenges  faced.  Furthermore,  discussion  of  the 
advantages  and  disadvantages  of  each  technique  is  conducted  so  as  to  understand  and  compare  the  efficiencies  with  the 
existing  techniques  based  on  the  strategies. 
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INTRODUCTION 

Artificial  intelligence  is  one  of  the  most  sorted  techniques  used  in  the  background  of  a  game  and  is 
considered  as  major  factors  for  fun  and  other  factors  in  commercial  computer  games.  Understanding  and 
developing  intelligent  systems  which  have  all  the  traits  of  human  is  the  aim  of  artificial  intelligence.  The  artificial 
intelligence  is  getting  smarter  day  by  day  in  virtual  worlds.  The  earliest  use  of  the  artificial  intelligence  in  games 
was  in  1952,  and  it  was  created  by  a  graduate  student  in  the  UK  which  helps  in  playing  a  perfect  game  of  tic-tac- 
toe.  At  present,  a  team  of  researchers  is  working  on  and  has  succeeded  in  creating  ATs  that  helps  in  defeating 
humans  in  more  complex  games.  By  developing  the  human  language  in  children  the  development  of  the  action, 
conceptualization  and  social  interaction  helps  in  providing  mutual  help.  Language  requires  bringing  together  of 
many  different  processes  and  draws  attention  to  the  need  for  an  interdisciplinary  approach  (Llobera  et  al.,  2017). 
So,  the  work  in  developmental  robotics,  cognitive  science,  psychology,  linguistics,  neuroscience,  engineering  is 
included. 

Data-driven  motion  synthesis  allows  animators  to  produce  convincing  character  movements  with  high- 
level  parameters.  The  various  techniques  that  make  use  of  large  motion  capture  datasets  and  machine  learning  to 
parameterize  motion  have  been  proposed  in  computer  animation  The  artificial  intelligence  data  approaches  needs 
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major  amount  of  data  preprocessing  which  also  includes  motion  segmentation,  alignment  and  labelli  So  preprocessing  is 
carefully  performed  by  the  human  intervention  and  the  results  obtained  appear  smooth  and  natural  by  making  full 
automation  difficult  and  thereby  require  technical  developers  to  maintain.  Continuous  based  motion  mechanism  is  required 
for  the  interactive  character  control  and  the  motion  graphs  are  effective  for  such  purpose  (Arikan  and  Forsyth  2002;  Lee  et 
al.  2002;  Kovar  et  al.  2002)  .  To  enhance  the  data  set  the  motion  graphs  replay  captured  motion  data  and  techniques  are 
used  in  the  same  class. (Min  and  chai.  2012).  The  motions  in  the  graphs  need  to  be  classified,  segmented  and  aligned  by 
applying  motion  blending  for  motion  synthesis  during  interactive  character.  Even  Though  (Kovar  and  Gleicher  .  2004)  try 
to  execute  the  process  the  choice  of  the  distance  metric  between  motions  and  segmentation  in  motion  sequence  which 
directly  affects  the  performance  and  accuracy.  Furthermore,  the  framework  mixes  all  motions  present  nearby  for 
incorporating  realistic  movements  without  any  motion  segmentation  and  classification.  Techniques  based  on  the 
reinforcement  learning  are  used  at  each  step  following  the  user  instruction.  (Safonova  and  Hodgins.  2007). 
Precomputation  is  performed  in  reinforcement  learning  which  increases  exponentially  concerning  the  number  of  actions 
available  to  the  character.  Levine  et  al.,  (2012)  reduced  the  dimensionality  by  dividing  the  motion  dataset  into  different 
classes.  The  reinforcement  learning  is  avoided  and  avoids  the  high  level  commands  provided  by  the  user  to  the  low-level 
motion  features  and  thus  the  mapping  is  resolved  by  a  number  of  parameters  in  reinforcement  learning. 

CHALLENGES  IN  REAL  TIME  STRATEGY  GAME 

The  military  unit  as  an  application  used  in  real-time  strategy  games.  This  type  of  game  contains  screen  which  has 
mapped  area  and  contains  buildings,  units  and  terrain.  Usually,  several  players  are  present  in  an  RTS  game. 
Other  than  the  players  there  are  various  game  entities  called  participants,  units  and  structures.  In  recent  years,  it  is  shown 
that  these  games  pose  a  challenge  to  artificial  intelligence  (Sethy  et  al.,  2015).  The  design  of  the  artificial  intelligence 
players  for  these  games  which  are  capable  of  beating  average  human  players  are  challenging  because, 

•  RTS  games  usually  control  the  large  number  of  units  which  can  be  controlled  simultaneously. 

•  RTS  games  are  real  time. 

•  Real  strategy  games  involve  attributes  such  as  observability,  actions  which  further  depict  instructions  on  the  Al 
techniques  which  is  later  deployed. 

RTS  games  own  a  variety  of  military  units  used  by  the  players  to  wage  war  and  uses  units  and  structures  to  aid  in 
resource  collection,  unit  production.  According  to  the  game  while  playing  the  match  players  must  balance  the  development 
of  the  economy,  infrastructure  associated  with  the  military  units.  They  also  contain  the  available  units  to  easily  attack  and 
defend  in  the  present  and  more  resources  and  upgrades  to  win  later.  Thereby  they  also  decide  which  units  and  structures  to 
produce  and  the  technologies  to  be  used  all  through  the  game  to  capture  the  appropriate  composition  of  groups  at  the 
correct  times  to  have  access  to  the  proper  balance  of  groups  at  the  appropriate  times.  Recent  RTS  games  have  stimulated 
the  creation  of  new  boats  with  new  concepts  and  architectures  and  led  to  increased  number  of  publications  addressing  open 
Al  problems  in  RTS  games.  These  types  of  games  have  long  term  goals  and  needs  greater  levels  of  abstraction  and 
reasoning.  Further,  it  contains  a  vast  space  of  actions  and  game  states  with  actions  which  has  long  term  effects  throughout. 
Considering  the  theoretical  point  of  view,  the  differences  between  the  RTS  games  and  traditional  games  such  as  chess  are; 

•  Players  can  issue  actions  at  the  same  time  (simultaneous  moves),  and  most  actions  take  some  amount  of  time  to 
complete  (durative  actions).  While,  in  Chess,  moves  alternate  between  players  and  the  actions  are  instantaneous. 
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•  RTS  games  are  real-time,  which  means  players  do  not  need  to  wait  for  their  opponents  to  go  to  the  next  move,  for 

example,  StarCraft  is  updated  at  24  iterations  per  second,  hence  players  can  issue  a  new  move  every  42  ms. 

Considering  in  Chess,  players  may  have  several  minutes  to  decide  the  next  move. 

•  Most  RTS  games  are  partially  observable  due  the  fog-of-war  that  only  lets  observe  the  area  of  the  map  under  the 

sight  range  of  the  player’s  units,  while  games  like  Chess  are  perfect  information  games. 

•  Most  RTS  games  are  non-deterministic,  since  some  actions  might  have  a  chance  of  failure,  and  some  other  actions 

might  have  multiple  possible  outcomes. 

Synnaeve  and  Bessiere  (2016)  tackle  the  problems  of  having  partial  information  and  the  inherent  computational 
complexity  of  RTS  games.  So  proposed  a  Bayesian  modelling  framework  that  can  handle  the  game  at  three  interconnected 
levels  of  abstraction  doing  interference  at  each  level. 

Chung,  Buro,  and  Schaeffer  (2005)  created  a  capture  the  flag  game  which  depicts  that  the  player  is  needed  to 
control  the  group  of  units  to  direct  through  the  obstacles  to  the  other  side  of  the  map  and  retrieve  the  opponent's  flag. 
Later  a  generalized  Monte  Carlo  planning  framework  and  then  applied  it  to  the  game,  giving  positive  results. 

The  challenges  that  are  faced  by  the  real  time  gaming  systems  are  briefed  below. 

Adversarial  Real-Time  Planning 

Domains  in  which  agents  compete  against  one  another  are  termed  adversarial  (Willmott  et  ah,  2001).  As  it  is  well 
known  that  the  real  time  strategy  games  and  its  actions  are  performed  in  real  time,  so  that  the  players  take  decisions  under 
major  restriction  and  executes  continuously.  Turn  based  game  agents  considers  the  environments,  but  the  fact  is  that  the 
game  is  not  happening  in  real  time  which  makes  challenge  difficult  to  avoid.  As  the  sizes  of  the  state  space  and  the  action 
space  are  much  larger  than  the  existing  ones,  a  game  tree  search  approaches  are  not  directly  applicable  due  to  a  large  game 
tree  to  explore.  One  of  the  best  available  solutions  is  using  multiple  levels  of  abstraction  and  perform  an  adversarial  game 
tree  search  in  few  problems  such  as  long-term  planning  or  short-term  planning.  As  the  abstractions  are  based  on  a  time  the 
temporal  reasoning  is  one  of  the  important  elements  to  be  considered.  In  case  of  high-level  representive,  the  combat 
between  the  models  needs  to  look  into  as  there  are  large  armies  (Alberto  Uriarte,  2017). 

Decision  Making  Under  Uncertainty 

Considering  most  of  the  RTS  games,  NPC  has  all  the  information  regarding  the  game  state  such  as  a  location  of 
the  player  units  and  buildings.  The  situation  is  considered  as  cheating  as  the  players  doesnt  have  information  and  to 
overcome  this  situation  the  imposition  of  the  observability  in  RTS  games,  making  sure  that  all  the  players  play  the  game  on 
equal  terms.  This  problem  doesn't  occur  in  turn  based  games  as  the  player  has  the  location  information  including  game 
states. 

Opponent  Modeling 

The  concept  is  that  the  players  have  strength  to  analyze  the  ability  of  the  enemies  actions  and  find  out  the 
weakness  which  exploits  the  future  games.  So  the  artificial  games  needs  to  be  cleared  to  find  the  opponent’s  plans  and 
identify  their  behaviors  on  basis  of  previous  observations  and  the  challenge  is  not  an  exclusive  in  RTS  games  as  in  any 
other  games  modeling  the  opponent  will  be  useful. 
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Spatial  and  Temporal  Reasoning 

During  the  game  play  the  maps  are  used  as  the  components  with  more  popularity.  Agents  can  develop  better 
offensive  plans  and  more  organized  resources  gathering  as  well.  The  main  advantage  of  the  human  players  is  that  they  have 
excellent  knowledge  in  temporal  relations  of  actions. 

Resource  Management 

RTS  games  develop  new  technologies  and  also  new  resources  which  are  used  to  upgrade  buildings  and  units. 
The  resources  or  the  attributes  present  is  spread  throughout  the  map  and  further  it  is  combined  obtained  for  buildings 
placed  in  certain  places  on  the  map.  Therefore,  for  the  successful  strategy  the  proper  resource  management  strategy  is 
considered. 

Collaboration 

During  an  RTS  game,  each  player  generates  many  units  and  thus  these  units  are  considered  as  a  multi  agent 
system  and  thus  it  is  important  to  develop  coordination  methods  that  lead  to  a  good  team  tactics  and  strategies. 
There  is  not  only  a  collaboration  at  unit  level,  but  also  team  matches  which  havetwo  or  more  team  players. 

Pathfinding  and  Content  Generation 

Finding  out  the  suitable  paths  in  an  easy  manner  between  the  two  locations  in  a  map  is  considered  as  one  of  the 
important  tasks  in  RTS  games.  Many  moving  objects  are  considered  while  calculating  paths  as  the  game  environment  is 
dynamic.  It  also  contains  more  aspects  such  as  keeping  unit  information,  taking  terrain  properties. 

Game  content  contains  all  matters  of  game  that  affects  the  game  play  than  playing  with  the  non  player  character 
behavior  and  also  the  game  engine.  In  RTS  games  the  content  is  assorted  such  as  maps,  units,  buildings 
(Cabrera  et  al.,  2013) 

CHALLENGES  IN  BUILDING  HUMAN  LIKE  MACHINES 

The  toughest  task  for  machine  learning  and  artificial  intelligence  is  learning  simple  video  concepts 
(Lake  et  ah,  2015). 

The  Characters  Challenge 

The  handwritten  character  recognition,  which  is  one  of  the  main  challenges  which  becomes  a  problem  for 
comparing  different  types  of  machine  learning  algorithms.  In  this,  handwritten  character  recognition  is  a  major  problem 
and  this  also  simpler  than  more  general  forms  of  object  recognition.  People  are  great  learners  than  the  algorithms  for 
concern  regards  character  recognition.  People  try  learning  more  and  attaining  the  human  level  learning  abilities  in 
machines  which  are  called  as  character  challenge.  People  can  efficiently  integrate  across  multiple  examples  of  a  character 
to  infer  which  have  optional  elements.  The  progress  continues  by  combining  the  deep  learning  and  probabilistic  program 
induction  to  tackle  the  higher  versions  of  character  challenge. 
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Figure  1:  The  Character  Challenge:  Human  Level  Learning  of  Handwritten  Characters 
The  Frostbite  Challenge 

The  second  challenge  concerns  the  Atari  game  frostbite,  which  was  considered  as  one  of  the  control  problems 
(Minlt  et  al.,  2015).  In  this,  the  network  is  trained  to  play  49  classic  Atari  games  proposed  as  test  domain  for  the 
reinforcement  learning  (Bellemare  et  al.,  2013).  In  frostbite,  players  control  an  agent  tasked  with  constructing  an  igloo 
within  the  time  limit.  The  igloo  is  built  piece  by  piece  as  the  agent  jumps  on  ice.  The  challenge  is  that  the  ice  floes  are  in 
stagnant  motion  and  only  the  ice  floes  contribute  to  the  construction  of  the  igloo.  The  agent  may  also  earn  extra  points  by 
gathering  fish  in  the  water  and  thereby  avoiding  major  hazards. 
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Figure  2:  Screenshots  of  Frostbite,  a  Video  Game  Designed  for  Atari  Game  Console 
TASKS  IN  REAL  TIME  ENVIRONMENT 

Because  of  the  characteristics,  RTS  games  have  many  challenges  to  deal.  The  below  shows  the  problems  to  be 
faced  in  real-time  environment: 

Planning 

Humans  consider  few  actions  to  form  the  strategy  and  that  is  considered  as  plan.  Planning  is  considered  as  the 
process  of  determining  the  action  sequences  which  accomplishes  a  goal  when  the  task  is  executed.  The  planning  process 
becomes  quite  difficult  because  of  the  presence  of  contender  in  addition  to  real  time  and  hidden  information.  Aha  et  al., 
(2013)  proposed  a  plan  retreval  algorithm  where  three  key  sources  of  domain  knowledge  are  used  along  with  the  removal 
of  the  assumption  of  a  static  component.  .  Moore  et  al.,  (2008)  introduced  an  integrated  algorithm  called  RL/CBR  which 
uses  continuous  models  instead  of  discrete  approximation.  The  algorithm  is  named  as  continuous  action  and  state  space 
learner  (CASSL).  Alcazar  et  al.,  (2008)  PDDL  was  used  to  define  a  planning  domain  that  is  used  to  implement  an  artificial 
player  based  on  automated  planning  in  a  RTS  game. 


www.tjprc.ors 


editor@tjprc.  org 


6 


Amitvikram  Nawalagatti  &  Prakash  R.  Kolhe 


Plan  Recognition  and  Predictions 

The  process  of  an  agent  observing  the  actions  of  the  other  agent,  whether  it  is  human  or  computer  based  is  called 
as  plan  recognition..  There  are  several  types  of  plan  recognition,  one  among  them  is  case-based  plan  recognition  which 
collects  the  data  on  the  building  construction  and  is  utilized  for  analysis  and  divide  the  players’  strategies  and  style. 
The  probabilistic  models  of  opponent  behavior  and  actions  were  learned  from  the  sets  of  saved  games.  Plan  recognition  can 
be  categorized  into  two  levels,  such  as  strategic  and  tactical.  Strategic  plans  dictate  the  kind  of  units  the  player  executes 
and  the  tactical  plans  dictate  the  units  deployed. 

HUMAN-MACHINE  INTERFACE 

The  neural  network  modeling  is  performed  and  the  approaches  required  for  the  human  intelligence  is  explained 
and  are  as  follows. 

Development  Start-Up  Software 

Humans  have  a  foundational  understanding  of  several  core  domains  (Spelke  and  kinzler,  2007).  The  domains 
consist  of  number,  space,  physics  and  psychology.  The  core  domains  cleave  cognition  at  its  conceptual  joints  and  the 
particular  domain  is  organized  by  a  set  of  entities  and  principles  relating  the  entities.  Each  core  domain  has  become  the 
target  of  a  great  deal  of  study  and  analysis.  All  of  the  domains  are  important  for  the  machine  learning. 

Intuitive  Physics 

The  promising  recent  approach  sees  intuitive  physical  reasoning  as  similar  to  inference  over  a  physics  software 
engine  (Bates  et  al.,  2015).  According  to  the  hypothesis,  people  reconstruct  the  scene  using  representations  of  the  objects 
and  few  properties  and  also  the  forces  acting  on  objects.  The  intuitive  physics  engine  approach  enables  flexible  adaptation. 
It  is  quite  difficult  to  integrate  object  and  physics-based  primitives  into  deep  neural  networks. 

Intuitive  Psychology 

Intuitive  psychology  is  another  early  emerging  ability  which  is  influenced  by  human  learning  and  thought. 
Animate  agents  were  separated  from  inanimate  objects.  The  full  formal  account  of  intuitive  psychological  reasoning  needs 
to  include  representations  of  agency,  goals,  and  efficiency  and  reciprocal  relations.  There  are  several  ways  to  incorporate 
the  intuitive  psychology  into  contemporary  deep  learning  systems.  These  abilities  are  early  emerging  and  play  an  important 
role  in  human  learning  and  thought. 

MACHINE  LEARNING  TECHNIQUES 

A  system  will  learn  from  an  experience  when  it  improves  in  completing  a  determined  task  by  optimizing  its 
performance.  It  is  also  applied  in  the  optimization  of  the  Non-player  characters  Different  machine  learning  tools  and 
techniques  used  by  different  authors  are  as  follows; 

Naive  Bayes  Classifier 

This  type  of  machine  learning  is  based  on  the  Bayes  theorem.  This  is  a  type  of  learning  and  prediction  technique 
based  on  storing  a  series  of  attribute-class.  The  attributes  present  in  the  class  are  independent  from  each  other  (Millington 
et  al.,  2009).  The  reviewing  of  the  technique  is  made  on  the  characteristics  in  the  methods.  They  are  as  follows. 
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Artificial  Neural  Networks 

These  neural  networks  constitute  of  a  paradigm  of  learning.  These  are  composed  a  set  of  neurons  or  states  and  is 
defined  by  a  set  of  features.  The  systems  receive  a  group  of  inputs  and  generate  a  specific  output.  These  ANNs  emulate  the 
human  brain  behavior  and  is  formed  by  nodes  which  is  connected  together  to  form  a  network.  The  ANN  gets  information 
from  the  neighboring  neurons  and  gives  an  output  depending  on  its  weights  and  functions. 

Tan  et  ah,  (2013)  found  the  cons  of  spectrum  lack  and  inefficient  in  the  communication  networks  and  the  by 
introducing  new  solutions  using  ANN  by  replacing  the  current  frequency  allocation  system. 

Yang  et  ah,  (2103)  introduced  genetic  algorithms  for  the  design  of  the  cognitive  engine  and  in  addition  the  radial 
basis  function  network  is  used  to  adjust  the  parameters  of  the  system  to  adapt  for  the  environmental  changes. 

Support  Vector  Machines 

Support  vector  machines  are  set  of  supervised  algorithms  focused  on  data  analysis  and  pattern  recognition.  Data  is 
clustered  using  hyper  planes  in  a  multidimensional  space  that  helps  in  maximizing  the  separation  between  classes. 
This  clustering  is  dependent  on  decision  boundaries. 

These  are  used  for  learning  models  and  also  as  regression  analysis.  The  basic  idea  behind  the  SVM  is  detecting 
the  soft  boundary  of  a  given  set  of  samples  so  as  to  classify  new  points.  SVM  s  delivers  a  unique  solution  and  produce  the 
efficient  classifiers  with  high  prediction  accuracy.  As  SVMs  are  inseparable  the  input  features  are  mapped  into  high 
dimensional  feature  space  (Abbas  et  ah,  2015).  Dandan  et  ah,  (201 1)  found  that  SVM  can  be  used  for  spectrum  sensing  and 
real  time  detection..  SVM  provides  high  performance  in  minor  problems,  but  the  complexity  increases  in  large  problems. 
SVMs  are  considered  as  computationally  expensive. 

Genetic  Algorithms 

The  genetic  algorithms  are  originated  by  mutating  small  FORTRAN  programs.  Genetic  algorithms,  search  in 
space  with  the  aim  of  finding  an  element  that  helps  in  maximizing  the  fitness  function  by  evolving  the  population  of 
chromosomes. 

Chen  et  ah,  (2010)  toof  genetic  algorithms  by  enhancing  the  system  performance  thus  by  solving  the  multi 
objective  problems  which  help  in  reducing  the  error  rate  and  power. 

Hauris  et  ah,  (2007)  proposed  genetic  algorithm  for  RF  parameter  optimization  in  CRR.  The  genes  used  are 
modulation  and  coding  schemes,  antenna  parameters,  transmit  and  receive  antenna  gains. 
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COMPARATIVE  ANALYSIS 


Table  1 


SI  No 

Title 

Technique  Used 

Outcome 

Research  Gap 

1. 

Review  of  the  Use  of  AI 
Techniques  in  Serious  Games: 
Decision-Making  and 

Machine  Learning 

Naive  Bayes 
classifier 

Non- 

deterministic. 

It  requires  large 
number  of  data 
records 

2. 

Review  of  the  Use  of  AI 
Techniques  in  Serious  Games: 
Decision-Making  and 

Machine  Learning 

Artificial  neural 
networks 

Adaptation 
ability  to  new 
changes  and 
complexity  is 
reduced. 

Poor 

generalization 
and  requires 
training  data 
labels 

3. 

Recent  advances  on  artificial 
intelligence  and  learning 
techniques  in  cognitive  radio 
networks 

Support  vector 
machines 

Generalizatio 
n  ability  and 
the  robustness 
against  noise. 

Complex  with 
the  large 
problems 

4. 

Recent  advances  on  artificial 
intelligence  and  learning 
techniques  in  cognitive  radio 
networks 

Genetic 

algorithms 

Multi¬ 

objective 

optimization 

High  complexity 
with  the  large 
problems  and 
also  require  prior 
knowledge  of 
the  system. 

CONCLUSIONS 

This  paper  presents  a  review  of  the  artificial  intelligence  based  machine  learning  techniques  for  designing 
interactive  characters.  The  table  above  gives  the  comparison  techniques  discussed  in  the  review.  The  machine  learning 
techniques  used  are  naive  Bayes  classifier,  support  vector  machines,  genetic  algorithms  and  neural  networks. 
The  techniques  used  to  create  a  new  intelligent,  serious  game  and  to  provide  the  players  with  a  real  experience. 
Further  comparing  the  techniques  it  is  observed  each  technique  has  its  own  pros  and  cons  whereas  neural  networks  are 
better  compared  to  other  techniques  and  thus  help  in  designing  of  the  player  characters  in  video  games.  Also,  future  work 
is  to  extend  the  state  of  the  art  in  the  field  of  games  by  creating  a  knowledge  hub  and  also  to  address  more  AI  techniques. 
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